Remote Direct Memory Access (“RDMA”) is a communications technique that allows data to be transmitted from the memory of one computer to the memory of another, remote computer without the data passing through either computer's central processing unit (“CPU”) and without calling an operating system kernel. RDMA was developed largely in response to increasing demands for network speed. Generally speaking, data can be transferred faster when the data does not have to pass through a CPU.
The InfiniBand™ computing architecture, for example, enables RDMA. InfiniBand™ has gradually been replacing the previously standard Peripheral Component Interconnect (“PCI”) bus architecture in certain servers and personal computers. While the PCI architecture sends data in parallel, the InfiniBand™ architecture sends data in serial, but concurrently carries multiple channels of data in a multiplexed signal.
In order to allow programmers to code their programs in such a way that their programs can reap the benefits of RDMA, industry groups have collaborated and produced the User Direct Access Programming Library (“UDAPL”). UDAPL is an Application Programming Interface (API) that defines methods that a program can invoke in order to perform RDMA operations. For example, UDAPL defines a method that, when invoked, causes a computer to read from a remote memory region. For another example, UDAPL defines a method that, when invoked, causes a computer to write to a remote memory region.
UDAPL operations are asynchronous. Much like the way a person might deposit a letter into a mailbox for delivery and then go on with life without waiting for a response to that letter, a program that invokes a UDAPL method may go on performing other operations without waiting for any acknowledgement that the operations that should have been performed by that method actually were performed. However, because programs often need some assurance that specified operations actually were performed prior to performing other operations, UDAPL implements an event system.
Under the event system, events are automatically generated in response to the completion of UDAPL operations. When an event is generated, an event message is automatically inserted into an event dispatcher queue. A program interested in knowing whether a UDAPL operation completed is responsible for determining whether the corresponding event message was placed in the event dispatcher queue. The UDAPL API provides methods that a program can invoke in order to help make this determination.
One such method, “dat_evd_dequeue( ),” causes the invoking process to poll the event dispatcher queue in an attempt to discover whether the queue contains a specified event message. While the process is polling the event dispatcher queue, other processes are precluded from using the CPU. If a process continuously invokes “dat_evd_dequeue( )” until a desired event message is found within the event dispatcher queue, then other concurrently executing processes will be forced to wait until the desired event message is found before the CPU can perform any more of those other processes' operations. This monopolization of the CPU may be unacceptable in many multiprocessing and/or multi-user scenarios.
Another method provided by UDAPL, “dat_evd_wait( ),” causes the invoking process to “block” until specified criteria, passed as a parameter to the method, are satisfied. When a process begins blocking, the operating system may store that process' in-memory data to a hard disk drive. While the process is blocking, the CPU may perform other process' operations; thus, the blocked process does not monopolize the CPU. When the specified criteria are satisfied and the process ceases blocking, the operating system may load the process' data into volatile memory from the hard disk drive so that the CPU can continue operating on that data. This loading is part of what is called a “context switch.” Although the “dat_evd_wait( )” method avoids CPU monopolization, the context switch that is sometimes performed when blocking ceases often requires a substantial amount of time to perform. As a result, the performance of the process that was blocking can suffer. Some “real-time” applications cannot tolerate significant quantities of such performance penalties.
Thus, the use of either of these UDAPL event-handling methods has an associated drawback. Previous approaches for determining whether a UDAPL event was completed tended to suffer either from CPU monopolization or performance penalties related to context switching.